
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 

Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 223)3-1450 
www.uspto.gov 



APPLICATION NO. 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 


10/685.162 


10/14/2003 


Anurag Mittal 


2002P 17242 USOl 


4275 



7590 08/09/2007 

Siemens Corporation 
Intellectual Property Department 
170 Wood Avenue South 
Iselin, NJ 08830 



EXAMINER 



MOTSINGER, SEAN T 



ART UNIT 



2624 



PAPER NUMBER 



MAIL DATE 



DELIVERY MODE 



08/09/2007 PAPER 

Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 



PTOL-90A (Rev. 04/07) 



Office Action Summary 


Application No. 

10/685,162 


Appllcant(s) 

MITTAL ET AL 


Examiner 
Sean Motsinger 


Art Unit 

2624 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address ~ 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1. 136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the nriailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three nrionths after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

I) D Responsive to communication(s) filed on 3/19/2007 , 

2a)l3 This action is FINAL. 2b)n This action is non-final. 

3) n Since this application is in condition for allowance except for fonnal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) 13 Claim(s) 1-13 and 25 is/are pending in the application. 

4a) Of the above claim(s) is/are wrthdrawn from consideration. 

5) 13 Claimfe) 1,2 and 25 is/are allowed. 

6) S Claim(s) 3-13 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10)S The drawing(s) filed on 12/8/2003 is/are: a)n accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

II) 0 The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)n Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (0. 
a)n All b)n Some * c)n None of: 

1 .□ Certified copies of the priority docunients have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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Response to Applicants Arguments 

1 . The amendment to the claims filed on 3/19/2007 has been entered and made of 
record. 

Objection to the Drawings: 

2. Applicant has traversed the objection to the drawings, however, applicant's 
arguments are not persuasive. Applicant has argued using text from form paragraph 
6.22.01 inM.P.E.P. section 608.02(d). The examiner in the objection to the 
drawings did not use this form paragraph. This paragraph applies to detail described 
in the specification and makes no reference to claimed subject matter. Therefore this 
paragraph is clearly not relevant. 

3. The applicant also argues that section 601 .01 (f) of the M.P.E.P. states that a 
drawing for a method claim "is not necessary for the understanding of the invention 
under 35 U.S.C. 113 (first sentence)." Examiner has not made a rejection under 35 
U.S.C. 113 and therefore this does not appear to be relevant. 

4. 37 CFR 1 .83 (a) states clearly "The drawings in a non-provisional application 
must show every feature specified in the claims." The rule does not provide for any 
exceptions. Any future response other then an amendment adding all of the features 
not shown, or an argument clearly stating where the features listed in the objection 
can be found in the drawings will be considered intentionally non-responsive. 



Rejections of Claims: 
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5. Re claims 3-5, applicant has amended claim 3 to state that his higher 
dimensional space includes at least one flow dimension and one color dimension (as 
interpreted by the examiner in view of the 1 12 rejection below.) Applicants argues 
this is allowable with respect to examiners inclusion or allowable subject matter in 
the previous action. However examiners allowance was in reference to a probability 
distribution having 3 color components and 2 flow components. This claim does not 
claim a probability distribution but only claims that the higher dimensional space 
includes a color dimension, and an optical flow (a measurement of movement) 
dimension. Therefore any space that accounts for movement has "optical flow 
dimensions." 

6. Re claim 6-10, applicant has amended claim 6 to include having a substantially 
dynamic background such as a naturally occurring outdoor environment, and 
provided an argument stating that this amendment makes the claim patentable over 
the prior art. However, applicant's arguments fail to comply with 37 CFR 1 .1 1 1(b) 
because they amount to a general allegation that the claims define a patentable 
invention without specifically pointing out how the language of the claims. patentably 
distinguishes them from the references. Applicant only describes differences in 
applicant's dynamic background and Lee's dynamic background. There is no 
argument with regard to how Lee's is not "substantially dynamic" only to how it is not 
"substantially continually dynamic," and applicants example of a type of dynamic 



Application/Control Number 1 0/685, 162 Page 4 

Art Unit: 2624 

background is only an example and is not given patentable weight. Therefore the 
rejections to claims 6-10 are being maintained despite the amendment. 



Objections to the Drawings 

7. The drawings are objected to under 37 CFR 1 .83(a). The drawings must show 
every feature of the invention specified in the claims. Therefore, the following 
features must be shown or the feature(s) canceled from the claim(s). In Claim 3 
"evaluating uncertainties," and "combining said optical flow measurements, said 
intensities, and said uncertainties." In claim 4, "applying an error function," "applying 
an optical flow constraint," "deriving a motion estimate," and "comparing said motion 
estimate." In claim 6 "deriving flow covariance matrices," "deriving intensity 
covariance matrices," "applying relevance criteria," and "performing detection." In 
claim 10 "applying an optical flow constraint," "applying an error function" "deriving 
amotion estimate," "comparing said motion estimate," "deriving covariance matrices," 
"performing optimized detection," evaluating uncertainties," and "combining said 
optical flow measurements, said intensities, and said uncertainties". In claim 1 1 "a 
joint 5-dimensional space comprising 3 color components and 2 flow components." 
No new matter should be entered. 

8. Corrected drawing sheets in compliance with 37 CFR 1.121 (d) are required in 
reply to the Office action to avoid abandonment of the application. Any amended 
replacement drawing sheet should include all of the figures appearing on the 
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immediate prior version of the sheet, even if only one figure is being amended. The 
figure or figure number of an amended drawing should not be labeled as "amended." 
If a drawing figure is to be canceled, the appropriate figure must be removed from 
the replacement sheet, and where necessary, the remaining figures must be 
renumbered and appropriate changes made to the brief description of the several 
views of the drawings for consistency. Additional replacement sheets may be 
necessary to show the renumbering of the remaining figures. Each drawing sheet 
submitted after the filing date of an application must be labeled in the top margin as 
either "Replacement Sheet" or "New Sheet" pursuant to 37 CFR 1.121(d). If the 
changes are not accepted by the examiner, the applicant will be notified and 
informed of any required corrective action in the next Office action. The objection to 
the drawings will not be held in abeyance. 

Rejections Under 35 U.S.C. 112 Second Paragraph 

The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

9. Claims 3-13 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. 



Application/Control Number: 10/685,162 Page 6 

Art Unit: 2624 

10. Re claim 3-5 Applicant claims a "higher dimensional space which includes an 
optical flow component and a normalized color component." It is not clear how in a 
"higher dimensional space" a "component" is a "dimension" or how a normalized 
color. For the purposes of examination examiner will interpret the claim to read 
"...higher dimensional space which includes an optical flow dimension and a color 
dimension." 

11. Re claim 6, the phrase "such as" renders the claim indefinite because it is 
unclear whether the limitations following the phrase are part of the claimed invention. 
See MPEP § 2173.05(d). For the purpose of examination examiner has interpreted 
the claim not to include this limitation. 

12. Re Claims 11-13 these claims are now indefinite because of confusion generated 
by the amendment/arguments in regard to claim 3. Examiner previously believed 
that claim 1 1 intended to claim the probability distribution to contain 3 color 
components and 2 flow components. However now it is unclear if the components 
refer to the "probability distribution" or to the "5-dimensional space". For the purpose 
or examination examiner will examine under the assumption that the probability 
distribution comprises 3 color components and 2 flow components. 



Rejections Under 35 U.S.C. 103 
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The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



13. Claim 3-10 are rejected under 35 U.S.C. 103(a) as being unpatentable over Lee 
US PGPUB 2003/0058237 in view of Simoncelli, Eero, "Bayesian Multi-Scale 
differential Optical flow" Handbook of Computer Vision and Applications, Academic 
Press, Vol. 2 pp. 397-422. 

14. Re claims 3-6 Lee discloses a method for dynamic scene modeling and change 
detection applicable to motion analysis, comprising the steps of: making 
measurements of optical flow (see paragraph 69 line 8) and intensities (paragraph 
20 lines 6-7 RGB information is intensities) for said dynamic scene(see paragraph 6 

. line 1 1 if objects are moving the scene is dynamic); utilizing said optical flow 
measurements for capturing (i.e. detecting paragraph 69 line 7) and modeling 
(model maintenance paragraph 69 line 4 ) the dynamics of said scene (paragraph 
71 lines 5-8 this section describes modeling movement which is the dynamics of the 
scene); evaluating uncertainties in said measurements (see paragraph 39 a 
covariance is a measure of uncertainty); and combining said optical flow 
measurements (see paragraph 69 lines 3-9 the model is maintained based on 
optical flow), said intensities (see paragraph 39 the pixels modeled in the RGB color 
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space (i.e. using intensities)), and said uncertainties (of Intensity) (see paragraph the 
pixels are also modeled via covariance (i.e. uncertainty)) so as to develop a robust 
(see paragraph 69 this model can handle movement and therefore is robust) 
representation ( model see paragraph 69 line 2) of said scene (background 
paragraph 69 line 2) in a higher dimensional space (see paragraph 69 line 4 The 
background has multiple layers and therefore is higher dimensional) which includes 
an optical flow dimension (paragraph 69 shows that optical flow Is used in the model 
therefore has an "optical flow dimension") and a color dimension (see paragraph 39). 
Lee does not disclose derive a figure of uncertainty (covariance is a figure of 
uncertainty page 410 Irst paragraph) for optical flow measurement data. However, 
Simoncelli discloses deriving a figure of uncertainty (covariance Is a figure of 
uncertainty page 410 Irst paragraph) for optical flow measurement data. The 
motivation to combine is to take advantage of using a "efficient" gradient approach 
while allowing for more then "small displacements" when calculating optical flow, see 
page 398 paragraph 2. Therefor it would have been obvious to one of ordinary skill 
in the art to modify the invention of Lee with Simoncelli to reach the aforementioned 
advantage. 

15. Re claim 4 Simoncelli further discloses wherein said step of evaluating 

uncertainties in said measurements comprises the steps of: applying an optical flow 
constraint equation (equation 14.1 page 398) at a given point defined by a spatial 
location and time (page 398 last paragraph) to obtain respective constraints; 
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applying an error function (page 399 second paragraph) to combine said respective 
constraints from each said given point (page 398 second paragraph) within a defined 
region for deriving a characteristic function(equation 14.3 page 399) ; deriving a 
motion estimate(velocity estimate 4*^ paragraph page 399) from said characteristic 
function; and comparing said motion estimate with a given uncertainty model 
(covariance information page 410 Irst paragraph) so as to derive a figure of 
uncertainty (indicate more uncertain vectors page 410 Irst paragraph) for optical 
flow measurement data. 

16. Re claim 5, Simoncelli further discloses wherein said figure of uncertainty (page 
400 4*^ paragraph) exhibits a Gaussian characteristic (page 401 paragraph 2). 

17. Re claim 6 Lee discloses a method for dynamic scene modeling and change 
detection applicable to motion analysis, comprising the steps of: deriving optical flow 
measurement data for a dynamic scene (Paragraph 69 line 9) having a substantially 
dynamic background (paragraph 70 note the background is moving) such as 
naturally occurring outdoor environmental conditions (note this appears to be an 
example and not given patentable weight); deriving intensity (RGB information) data 
for said dynamic scene (paragraph 20 lines 5-8); deriving intensity covariance 
matrices for said intensity data (paragraph 39 line 3); applying relevance criteria 
(training see paragraph 57 line 1) to said intensity covariance matrices (paragraph 
57 line 5 note the parameters of the model included covariance matrices) for 



Application/Control Number: 1 0/685, 1 62 Page 1 0 

Art Unit: 2624 

selecting relevant features (parameters paragraph 57 line 4) of a subset thereof; and 
combining said relevant features (into a trained model paragraph 57 line 5 ) of said 
subset for performing detection (background foreground segmentation paragraph 
52). Lee does not disclose, Deriving flow covariance matrices for said optical flow 
measurement data; applying relevance criteria to said flow covariance matrices for 
selecting relevant features of a subset thereof and combining said relevant features 
of said subset for performing detection. However Simoncelli discloses deriving flow 
covariance matrices for said optical flow measurement data (paragraph 3 page 401 ); 
applying relevance criteria (uncertainty of vectors page 410 first paragraph) to said 
flow covariance matrices for selecting relevant features (uncertainty of vectors page 
410 first paragraph) of a subset thereof and combining said relevant features of said 
subset for performing detection (these features are used for optical flow which is 
used in detection of background layers of Lee paragraph 69). The motivation to 
combine is to take advantage of using a "efficient" gradient approach while allowing 
for more then "small displacements" when calculating optical flow, see page 398 
paragraph 2. Therefor it would have been obvious to one of ordinary skill in the art to 
modify the invention of Lee with Simoncelli to reach the aforementioned advantage. 

18. Re claim 8 Lee further discloses wherein said step of combining said relevant 
features comprises detection at a particular pixel (note the segmentation is 
preformed on each pixel paragraph 52 line 2). 
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1 9. Re claim 9 Simoncelli further discloses wherein said step of deriving optical flow 
measurement data comprises applying the Simoncelli algorithm (see title and 
author.) 

20. Re claim 10 Simoncelli further discloses, wherein said step of deriving optical 
flow measurement data comprises the steps of: applying an optical flow constraint 
equation (equation 14.1 page 398) at a given point defined by a spatial location and 
time (page 398 last paragraph) to obtain respective constraints; applying an error 
function (page 399 second paragraph) to combine said respective constraints from 
each said given point (page 398 second paragraph) within a defined region for 
deriving a characteristic function(equation 14.3 page 399) ; deriving a motion 
estimate(velocity estimate 4"^ paragraph page 399) from said characteristic function; 
and comparing said motion estimate with a given uncertainty model (covariance 
information page 410 1rst paragraph) so as to derive a figure of uncertainty 
(covariance is a figure of uncertainty page 410 Irst paragraph) for optical flow 
measurement data; deriving flow covariance matrices for said optical flow 
measurement data (paragraph 3 page 401); evaluating uncertainties in optical flow 
measurements (indicates uncertain vectors page 410 paragraph 1.) While lee further 
discloses, deriving intensity covariance matrices for said intensity measurement data 
(see paragraph 39); evaluating uncertainties in said intensity measurements (see 
paragraph 39 a covariance is a measure of uncertainty.) combining said optical flow 
measurements (see paragraph 89 lines 3-9 the model is maintained based on 
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optical flow), said intensities (see paragraph 39 the pixels modeled in the RGB color 
space (i.e. using intensities)), and said uncertainties (of intensity) (see paragraph the 
pixels are also modeled via covariance (i.e. uncertainty)) so as to develop a robust 
(see paragraph 69 this model can handle movement and therefore is robust) 
representation ( model see paragraph 69 line 2) of said scene (background 
paragraph 69 line 2) in a higher dimensional space (see paragraph 69 line 4 The 
background has multiple layers and therefore is higher dimensional). 

21 . Claim 7 Is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Simoncelli and Lee as applied to claim 6 above, and further in view of "Statistical 
modeling and performance characterization of a real-time dual camera surveillance 
system" Greiffenhagen, M.; Ramesh, V.; Comaniciu, D.; Niemann, H.; Computer 
Vision and Pattern Recognition, 2000. Proceedings. IEEE Conference on Volume 2, 
13-15 June 2000 Page(s):335 - 342. 

22. Re claim 7, Lee and Simoncelli discloses all of the features of claim 6. They do 
not disclose wherein said step of deriving intensity covariance matrices comprises a 
step of deriving covariance matrices of normalized color representation. However 
Greiffenhagen discloses wherein said step of deriving intensity covariance matrices 
comprises a step of deriving covariance matrices (in page 3 column 1 first paragraph 
2 line 15) of normalized color representation (illumination invariant in page 3 column 
1 first paragraph 2 line 15), The motivation to combine found in page 3 column 1 first 
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paragraph is to "handle shadowing and illumination changes." Therefore one of 
ordinary skill in the art would find it obvious to combine Griffenhagen with Lee and 
Simoncelli to reach the aforementioned advantage. 

Allowable subject Matter 

23, Allowable subject matter, claims 1-2 and 25 are allowed. Claims 1 and 25 are 
allowable because the prior art does not include adding said current input to said 
image background model with a low probability whenever said probability is below 
said threshold. Claim 2 contains allowable subject matter because it depends from 
claim 1. 

24. Claim 1 1 would be allowable if rewritten or amended to overcome the rejection(s) 
under 35 U.S.C. 112, 2nd paragraph, set forth in this Office action. Claim. 1 1 
contains allowable subject matter because while Lee show discloses a probability 
distribution containing 3 color components and Simoncelli discloses a distribution 
with 2 flow components they are not in a joint 5-dimenstional space. Therefore the 
prior art does not show a probability distribution of said intensity and said optical 
flow in a joint 5-dimensional space comprising 3 color components and 2 flow 
components. Claims 12-13 contain allowable subject matter because the depend 
from claim 1 1 
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Conclusion 



Any inquiry concerning this communication or earlier communications from the 



examiner should be directed to Sean Motsinger whose telephone number is 571-270- 
1237. The examiner can normally be reached on 9-5 M-F. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jingge Wu can be reached on (571 )272-7429. The fax phone number for 
the organization where this application or proceeding Is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 671-272-1000. 



Motsinger 



5/18/2007 




